Performance Optimization System for Hadoop and Spark Frameworks
نویسندگان
چکیده
منابع مشابه
Hadoop performance modeling and job optimization for big data analytics
Big data has received a momentum from both academia and industry. The MapReduce model has emerged into a major computing model in support of big data analytics. Hadoop, which is an open source implementation of the MapReduce model, has been widely taken up by the community. Cloud service providers such as Amazon EC2 cloud have now supported Hadoop user applications. However, a key challenge is ...
متن کاملMapReduce Frameworks: Comparing Hadoop and HPCC
MapReduce and Hadoop are often used synonymously. For optimal runtime performance, Hadoop users have to consider various implementation details and configuration parameters. When conducting performance experiments with Hadoop on different algorithms, it is hard to choose a set of such implementation optimizations and configuration options which is fair to all algorithms. By fair we mean default...
متن کاملPerformance Optimization of a Distributed Transcoding System based on Hadoop for Multimedia Streaming Services
In recent times, Hadoop based on the MapReduce model has gained considerable attention because the features of the data preprocessing techniques are not timeconsuming and are suitable for processing large-scale data. In particular, MapReduce is emerging as an important programming model for developing distributed dataprocessing applications such as web indexing, data mining, log file analysis, ...
متن کاملPre-stack Kirchhoff Time Migration on Hadoop and Spark
Pre-stack Kirchhoff time migration (PKTM) is one of the most widely used migration algorithms in seismic imaging area. However, PKTM takes considerable time due to its high computational cost, which greatly affects the working efficiency of oil industry. Due to its high fault tolerance and scalability, Hadoop has become the most popular platform for big data processing. To overcome the shortcom...
متن کاملDeveloping System Performance Metrics for Cloud Computing Based on Hadoop
This short white paper describes our efforts to establish techniques and tools to identify optimization opportunities for Hadoop workloads. Suitable performance metrics and relevant benchmark use cases are a crucial component to achieve these goals. We discuss efforts to define suitable metrics for cloud computing in general, briefly describe hardware and software components that impact Hadoop ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Cybernetics and Information Technologies
سال: 2020
ISSN: 1314-4081
DOI: 10.2478/cait-2020-0056